feat(nodes): add NVIDIA Nemotron LLM node - #1766
Conversation
Adds llm_nemotron: NVIDIA's Nemotron reasoning models via the OpenAI-compatible build.nvidia.com API, with self-hosted NIM/vLLM profiles, filtered sync_models integration, CI key wiring, tests, and docs. Closes #1765 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Leela8256
added
feature
🤖 Internal: Discord sync markerAuto-managed by the Discord notification workflow. Stores the linked Discord message ID and forum thread ID. Do not edit or delete. |
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughAdds an ChangesNemotron integration
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant Pipeline
participant IGlobal
participant Chat
participant NVIDIAAPI
Pipeline->>IGlobal: beginGlobal()
IGlobal->>Chat: create configured Chat
Chat->>NVIDIAAPI: invoke Nemotron with questions
NVIDIAAPI-->>Chat: return response content
Chat-->>Pipeline: return answers without reasoning blocks
Possibly related PRs
Suggested labels: Suggested reviewers: 🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (3 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@nodes/src/nodes/llm_nemotron/nemotron.py`:
- Around line 85-88: Update _THINK_BLOCK_RE and _chat so a <think> block is
removed through either its closing </think> tag or the end of results.content
when the response is truncated. Preserve normal response text after closed
blocks, and add a test covering an unterminated reasoning block at the token
limit.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: da564ac1-2807-46cb-a94b-f57a37e31092
⛔ Files ignored due to path filters (1)
nodes/src/nodes/llm_nemotron/nemotron.svgis excluded by!**/*.svg
📒 Files selected for processing (16)
.github/workflows/sync-models.ymldocs/README-nodes.mdnodes/src/nodes/llm_nemotron/IGlobal.pynodes/src/nodes/llm_nemotron/IInstance.pynodes/src/nodes/llm_nemotron/README.mdnodes/src/nodes/llm_nemotron/__init__.pynodes/src/nodes/llm_nemotron/nemotron.pynodes/src/nodes/llm_nemotron/requirements.txtnodes/src/nodes/llm_nemotron/services.jsonnodes/test/framework/pipeline.pytools/sync_models/README.mdtools/sync_models/src/providers/nemotron.pytools/sync_models/src/sync_models.config.jsontools/sync_models/src/sync_models.pytools/sync_models/test/markers.pytools/sync_models/test/test_sync_live.py
…erations A generation that hits max_tokens mid-reasoning emits no closing </think> tag; make the closing tag optional so partial reasoning never leaks downstream. Add unit tests for terminated, unterminated, multiple, and absent think blocks (stub-loaded, no heavy deps). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@nodes/test/test_llm_nemotron_think_strip.py`:
- Line 18: Add a concise PEP 257 docstring to the _load_nemotron helper
describing that it stubs dependencies and dynamically loads the Nemotron module
under test. Keep the helper’s behavior unchanged.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 00c45cff-cdc0-45ff-83c9-bd5b67d47127
📒 Files selected for processing (2)
nodes/src/nodes/llm_nemotron/nemotron.pynodes/test/test_llm_nemotron_think_strip.py
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
asclearuc
left a comment
asclearuc
left a comment
There was a problem hiding this comment.
Thanks @Leela8256 for this — it is a clean, well-scoped addition that mirrors the llm_kimi node template closely (profiles, the IGlobal save-time probe, the shared _format_error), ships four <think>-strip unit tests plus the engine mock test, and wires up the sync provider and CI secret in step with the existing providers.
Approving. One non-blocking should fix inline on nemotron.py about the _chat override dropping agent stop sequences (delegate to super()._chat()).
The one CodeRabbit 🟠 Major finding (unterminated <think> at max_tokens, #1766 (comment)) is already addressed at this head — the regex now matches (?:</think>|$) and a truncated-response test covers it — so it does not block.
Co-authored-by: Alexandru Sclearuc <asclearuc@gmail.com>
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
nodes/src/nodes/llm_nemotron/nemotron.py (1)
64-77: 🔒 Security & Privacy | 🟠 Major | ⚡ Quick winUse exact host validation before sending the NVIDIA API key.
At Line 76,
serverbaseis classified as a cloud endpoint with substring matching. A custom URL such ashttps://attacker.example/api.nvidia.com/v1can therefore receive the real NVIDIA key throughChatOpenAI. Uppercase and look-alike hosts are also misclassified.Parse the URL, allowlist the exact approved NVIDIA hostname and HTTPS scheme, and reuse the same predicate in
nodes/src/nodes/llm_nemotron/IGlobal.pyso validation and runtime use the same credential boundary.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@nodes/src/nodes/llm_nemotron/nemotron.py` around lines 64 - 77, Replace the substring check in the Nemotron credential guard around serverbase and apikey with a URL-based predicate that requires HTTPS and the exact approved NVIDIA hostname, rejecting attacker-controlled paths, look-alike hosts, and incorrect casing as cloud endpoints. Define or reuse this predicate in IGlobal.py and apply the same logic in nemotron.py so validation and ChatOpenAI runtime credential handling share one credential boundary.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@nodes/src/nodes/llm_nemotron/nemotron.py`:
- Around line 64-77: Replace the substring check in the Nemotron credential
guard around serverbase and apikey with a URL-based predicate that requires
HTTPS and the exact approved NVIDIA hostname, rejecting attacker-controlled
paths, look-alike hosts, and incorrect casing as cloud endpoints. Define or
reuse this predicate in IGlobal.py and apply the same logic in nemotron.py so
validation and ChatOpenAI runtime credential handling share one credential
boundary.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: a0cd06f9-f5a0-4a9c-8c62-f54f016a2bdd
📒 Files selected for processing (1)
nodes/src/nodes/llm_nemotron/nemotron.py
…tion The node's _chat now delegates the model call to ChatBase._chat (which forwards agent stop sequences); teach the test stub the same method so the tests exercise the delegated call path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
CI failure after e7ac4bf explained and fixed in 46cfe6e: the think-strip unit tests stub |
2 participants
Closes #1765
Summary
Adds
llm_nemotron, a dedicated LLM node for NVIDIA's Nemotron family — open-weight hybrid Mamba-Transformer reasoning models with up to 1M-token context — served through NVIDIA's OpenAI-compatible cloud API (build.nvidia.com,nvapi-keys) or self-hosted via NIM containers / vLLM / SGLang.What's in
Node (
nodes/src/nodes/llm_nemotron/)services.json:questions → answerslane; profiles for Nemotron 3 Super 120B (default), Ultra 550B, Nano 30B — all flaggedcapabilities.reasoning— plus a Custom profile and two self-hosted (Local) profiles. NIM containers serve the same model IDs as the hosted API, so local profiles reuse the cloud model strings againsthttp://localhost:8000/v1.IGlobal: save-time 1-token validation probe with structured OpenAI-exception → user-facing error mapping. Probe only runs for cloud endpoints; key presence (not format) is enforced — NVIDIA keys usenvapi-, notsk-.nemotron.py:Chat(ChatBase)vialangchain_openai.ChatOpenAI; strips<think>…</think>blocks so downstream nodes only see the final answer (the hosted API returns reasoning inreasoning_content, so the strip is a safety net for self-hosted servers).IInstance(LLMBase),__init__,requirements.txt, brand icon (two-color so the monochrome auto-theming leaves NVIDIA green intact), README with generated params block.Model sync (
tools/sync_models/)NemotronProvider+ registry entries + config block (ROCKETRIDE_NVIDIA_KEY)./v1/modelslists a full multi-vendor catalog (Llama, GLM, Kimi, Mistral, …), so the model filter includes onlynvidia/*nemotron*chat models and excludes vl/vision/omni/safety/guard/parse/ocr/embed/rerank/retriever/asr/tts/speech/audio/reward variants.Integration
ROCKETRIDE_NVIDIA_KEYmapped in thesync-modelsworkflow (partition + strict env blocks).requires_nvidiamarker + live profile-existence test.docs/README-nodes.md; provider row in the sync README.Testing
nodes/test/test_contracts.py: 316/316 pass (node contract + module import).builder nodes:test --pytest-pattern=llm_nemotron: engine-level run green (mocked LLM path)./v1/models; 1-token probe and real completions confirmed;test_nemotron_profiles_exist_in_apipasses with a real key.sync_models --provider llm_nemotron --model-source providerdry run reports no changes after the checked-in profiles.agent_langchain,agent_llamaindex),tool_chartjsLLM binding, nested sub-agent orchestration, multiplellm_nemotronnodes with different profiles in one pipeline, webhook-source pipelines, Custom-profile connections (cloud + custom base URL), 5-way concurrent requests on one instance, and clean surfaced errors for invalid keys (403) and unknown models (404). Self-hosted profile path verified against a local OpenAI-compatible mock server.ruff check/ruff formatclean; services JSONC validated.Follow-ups (not in this PR)
ROCKETRIDE_NVIDIA_KEYActions secret to enable native discovery/smoke-testing in the weekly sync (without it, the provider is enriched via fallback).🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
Documentation
Bug Fixes